Papers by Joshua M. Susskind
Regularized Training of Nearest Neighbor Language Models (2022.naacl-srw)
Copied to clipboard
| Challenge: | kNN-LM uses pre-trained language models and an exhaustive knn search to improve performance. |
| Approach: | They build upon kNN-LM, which uses a pre-trained language model and a knn search through the training data to achieve state-of-the-art results. |
| Outcome: | The proposed method improves on language modeling tasks on WIKI-2 and WIKI-103. |
Construction of Paired Knowledge Graph - Text Datasets Informed by Cyclic Evaluation (2024.lrec-main)
Copied to clipboard
Ali Mousavi, Xin Zhan, He Bai, Peng Shi, Theodoros Rekatsinas, Benjamin Han, Yunyao Li, Jeffrey Pound, Joshua M. Susskind, Natalie Schluter, Ihab F. Ilyas, Navdeep Jaitly
| Challenge: | Prior studies have shown that sequence-to-sequence models learn to hallucinate when the conditioning data has poor correlation with the sequence being produced. |
| Approach: | They construct a dataset that pairs Knowledge Graphs (KG) and text together and compare their results to a cyclic evaluation model. |
| Outcome: | The proposed model performs better on cyclic generation of KGs than on KG-T, but less well on synchronization of KTs. |